Skip to content

kueue: Add queue maintenance mode controls and drain actions - #1252

Open
abhayrajjais01 wants to merge 1 commit into
headlamp-k8s:mainfrom
abhayrajjais01:feat/kueue-queue-maintenance-controls
Open

abhayrajjais01 wants to merge 1 commit into
headlamp-k8s:mainfrom
abhayrajjais01:feat/kueue-queue-maintenance-controls

Conversation

@abhayrajjais01

Copy link
Copy Markdown

Description

This PR introduces interactive Queue Maintenance Mode controls (HoldAndDrain, Hold, and None resume) for ClusterQueue and LocalQueue resources in the Headlamp Kueue plugin.

In Kueue, cluster operators manage queue availability and maintenance via spec.stopPolicy. When taking nodes offline or performing cluster maintenance, operators need direct controls in Headlamp to:

  1. Pause Admissions (Hold): Safely pauses new workload admissions while allowing running workloads to complete normally.
  2. Drain Queue (HoldAndDrain): Pauses new admissions and evicts currently admitted workloads (with safety confirmation dialog).
  3. Resume Operations (None): Re-enables normal workload admission.

What Changes

  • Formatters & Helpers (src/resources/queueControlFormatters.ts):
    • Added getStopPolicyColor, getStopPolicyLabel, and getStopPolicyDescription to provide color-coded chips and operational explanations.
    • Added createStopPolicyPatch for standard Kubernetes spec.stopPolicy patch mutations.
  • Interactive UI Components (src/components/common/QueueMaintenanceControl.tsx):
    • QueueMaintenanceControl: Renders maintenance status badge with color coding (Green: Active, Orange: Paused (Hold), Red: Drain (HoldAndDrain)).
    • Quick action buttons with loading spinners and error alerts.
    • Confirmation modal for disruptive HoldAndDrain actions explaining that in-flight workloads will be evicted.
  • Integration:
    • Integrated QueueMaintenanceControl into ClusterQueueDetail and LocalQueueDetail views.
  • Unit Tests (src/resources/queueControlFormatters.test.ts):
    • Added 7 unit tests verifying color mapping, label generation, patch creation, and fallback handling.

@Ralthos Ralthos left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This closes a real gap: spec.stopPolicy is read-only on the LocalQueue page today. The
drain confirmation is the right call.

No access check on the controls

QueueMaintenanceControl goes into extraSections unconditionally, under the
KueueAdminResourceAccess that checks verb="get". A read-only user sees
Drain Queue (HoldAndDrain), confirms, and gets a 403. AuthVisible with authVerb="patch"
would hide the buttons.

LocalQueue also needs a namespace there, or the check asks about patching cluster-wide and
refuses anyone holding the permission through a RoleBinding. That is #1199 again.

cohortName breaks the empty case

-                  value: renderCohortLink(clusterQueue.spec.cohortName),
+                  value: renderCohortLink(clusterQueue.cohortName),

The getter returns this.spec.cohortName || '-', and renderCohortLink guards on
if (!cohortName). Since '-' is truthy, a ClusterQueue with no cohort now renders a link to a
cohort named -. Looks unrelated to the feature, so possibly accidental.

Six strings never reach the catalog

t(getStopPolicyLabel(...)) builds the key at runtime and i18next-parser extracts statically,
so the three labels and three descriptions are absent from translation.json. The parser needs
the literals at the t() call site.

The i18n array is empty

"headlamp": { "i18n": [] } lists no locales, so the locales/en/translation.json added here
may never load. prometheus lists 19, flux lists ["en"].

@abhayrajjais01
abhayrajjais01 force-pushed the feat/kueue-queue-maintenance-controls branch from 6fbc27d to 1f76e91 Compare August 31, 2026 11:30
@abhayrajjais01

Copy link
Copy Markdown
Author

This closes a real gap: spec.stopPolicy is read-only on the LocalQueue page today. The drain confirmation is the right call.

No access check on the controls

QueueMaintenanceControl goes into extraSections unconditionally, under the KueueAdminResourceAccess that checks verb="get". A read-only user sees Drain Queue (HoldAndDrain), confirms, and gets a 403. AuthVisible with authVerb="patch" would hide the buttons.

LocalQueue also needs a namespace there, or the check asks about patching cluster-wide and refuses anyone holding the permission through a RoleBinding. That is #1199 again.

cohortName breaks the empty case

-                  value: renderCohortLink(clusterQueue.spec.cohortName),
+                  value: renderCohortLink(clusterQueue.cohortName),

The getter returns this.spec.cohortName || '-', and renderCohortLink guards on if (!cohortName). Since '-' is truthy, a ClusterQueue with no cohort now renders a link to a cohort named -. Looks unrelated to the feature, so possibly accidental.

Six strings never reach the catalog

t(getStopPolicyLabel(...)) builds the key at runtime and i18next-parser extracts statically, so the three labels and three descriptions are absent from translation.json. The parser needs the literals at the t() call site.

The i18n array is empty

"headlamp": { "i18n": [] } lists no locales, so the locales/en/translation.json added here may never load. prometheus lists 19, flux lists ["en"].

Hi @Ralthos ,

Thanks for the great feedback! have updated the PR with all your suggestions:

Permission Check: Wrapped the buttons in <AuthVisible authVerb="patch"> and passed the namespace for LocalQueue so users with namespace-scoped permissions aren't blocked
Cohort Link: Fixed ClusterQueue details to pass spec.cohortName so empty cohorts show plain '-' instead of a broken link.
i18n Translations: Updated all status labels and descriptions to use literal strings so i18next-parser extracts them into translation.json.
package.json: Added "i18n": ["en"] to headlamp configuration.

happy to make any further changes if needed

@Ralthos

Ralthos commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

All four landed cleanly. Thanks for turning it around so fast.

The buttons sit inside AuthVisible with authVerb="patch", and the namespace does the
right thing for both types: it is undefined on a cluster-scoped ClusterQueue, so that check
stays cluster-wide, while LocalQueue gets its own.

Cohort reads spec.cohortName again, so an empty one renders a plain dash. The
cohortName === '-' guard is belt and braces now that the caller is fixed, and harmless either way.

Labels and descriptions are literals at the t() call site, all six are in
translation.json, and "i18n": ["en"] is set. Custom stop policy: {{policy}} is a nice
touch for a value neither map knows.

Formatter tests pass here and tsc is clean.

@abhayrajjais01
abhayrajjais01 force-pushed the feat/kueue-queue-maintenance-controls branch from 1f76e91 to 48fb378 Compare September 29, 2026 08:10
@illume
illume requested a balanced review from Copilot October 11, 2026 05:32

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Default ClusterQueue state handling and named-resource authorization checks are currently incorrect.

1 open finding
What changed in this PR

Adds maintenance-mode controls for Kueue ClusterQueues and LocalQueues.

Changes:

  • Adds pause, drain, resume, authorization, confirmation, and error handling.
  • Adds stop-policy formatting, localization, and unit tests.
  • Prevents placeholder cohort links.
File Description
kueue/​src/​resources/​queueControlFormatters.ts Adds stop-policy helpers.
kueue/​src/​resources/​queueControlFormatters.test.ts Tests formatter behavior.
kueue/​src/​components/​common/​QueueMaintenanceControl.tsx Implements maintenance controls.
kueue/​src/​components/​clusterqueues/​Detail.tsx Integrates ClusterQueue controls.
kueue/​src/​components/​localqueues/​Detail.tsx Integrates LocalQueue controls.
kueue/​src/​components/​common/​KueueResourceLinks.tsx Avoids placeholder cohort links.
kueue/​package.json Enables English localization.
kueue/​locales/​en/​translation.json Adds maintenance UI strings.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@illume illume left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this.

Would you mind addressing the open Copilot review comments? Please mark each comment as resolved after addressing it.

@abhayrajjais01
abhayrajjais01 force-pushed the feat/kueue-queue-maintenance-controls branch 2 times, most recently from b1809d8 to f095dcc Compare October 11, 2026 09:17
@illume illume added enhancement New feature or request kueue labels Oct 11, 2026
@illume
illume requested a balanced review from Copilot October 11, 2026 14:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The core maintenance workflow is sound; identified concerns are limited to minor UX and maintainability improvements.

3 open findings
1 resolved since last review

🧠 Review effort: Balanced

@illume illume left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this PR.

The open review comments from Copilot still need attention — can you have a look? Once addressed, please mark them as resolved.

@abhayrajjais01
abhayrajjais01 force-pushed the feat/kueue-queue-maintenance-controls branch from f095dcc to 05db3d4 Compare October 12, 2026 07:48
@illume
illume requested a balanced review from Copilot October 12, 2026 12:59

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The operational descriptions and drain warning omit reservation cancellation and overstate immediate eviction behavior.

2 open findings
3 resolved since last review

🧠 Review effort: Balanced

@abhayrajjais01
abhayrajjais01 force-pushed the feat/kueue-queue-maintenance-controls branch from 05db3d4 to c369e64 Compare October 13, 2026 09:36

@illume illume left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for these changes.

Can you please address the open review comments? Once you've resolved each one, please mark it as resolved.

@abhayrajjais01
abhayrajjais01 force-pushed the feat/kueue-queue-maintenance-controls branch from c369e64 to 2ba4b8b Compare October 13, 2026 09:45
@abhayrajjais01

Copy link
Copy Markdown
Author

@illume,
i have addressed all the suggestions from the copilot, some of them are not showing as resolved but i have fixed then you can take a look at the code changes

thank you, looking forward for your feedback

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Workload-evicting actions warrant human validation of authorization and confirmation flows, and misleading readiness wording remains unresolved.

1 open finding
2 resolved since last review

🧠 Review effort: Balanced

@illume illume left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for these changes.

Can you please address the open review comments? Once you've resolved each one, please mark it as resolved.

Implement QueueMaintenanceControl card with AuthVisible RBAC checks,
stopPolicy formatters, type-only imports, targeted action loading states,
and drain confirmation modal.

Signed-off-by: Abhayraj Jaiswal <abhayraj916146@gmail.com>
@abhayrajjais01
abhayrajjais01 force-pushed the feat/kueue-queue-maintenance-controls branch from 2ba4b8b to 52ff329 Compare October 18, 2026 16:30
@illume
illume requested a balanced review from Copilot October 19, 2026 06:18

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The workload-evicting drain flow needs human validation of authorization, confirmation, and API behavior across supported Kueue versions.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request kueue

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants